Server, server in the cloud. Who is the fairest in the crowd?

This paper follows the recent history of automated beauty competitions to discuss how machine learning techniques, in particular neural networks, alter the way attractiveness is handled and how this impacts the cultural landscape. We describe experiments performed to probe the behavior of two different convolutional neural network architectures in the classification of facial attractiveness in a large database of celebrity faces. As opposed to other easily definable facial features, attractiveness is difficult to detect robustly even for the best classification systems. Based on the observations from these experiments, we discuss several approaches to detect factors that come into play when a machine evaluates human features, and how bias can occur not only in data selection but in network architectures; in multiple forms on multiple levels throughout the process. The overall goal is to map out with mixed methods a novel condition: slippages produced by platform level machine learning systems that make judgements in domains considered dependent on high level human intelligence.